Papers with Transformer system
Shallow-to-Deep Training for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that deep training is 1:4 faster than training from scratch. |
| Approach: | They propose a shallow-to-deep training method that learns deep models by stacking shallow models. |
| Outcome: | The proposed method is 1:4 faster than training from scratch and achieves BLEU scores of 30:33 and 43:29 on two translation tasks. |
Towards Linear Time Neural Machine Translation with Capsule Networks (D19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) is an endto-end learning approach to machine translation. |
| Approach: | They propose a capsule network with dynamic routing for linear time Neural Machine Translation . they map the source sentence into a matrix with pre-determined size and apply a deep LSTM network to decode the target sequence from the source representation. |
| Outcome: | The proposed network achieves comparable results with the Transformer system on English-German and English-French tasks. |
Learning Deep Transformer Models for Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural machine translation models have advanced the previous state-of-the-art by learning mappings between sequences via neural networks and attention mechanisms. |
| Approach: | They propose to use layer normalization to pass the combination of previous layers to the next layer to improve the model. |
| Outcome: | The proposed model outperforms the shallow Transformer-Big/Base baseline model on English-German and Chinese-English tasks by 0.4-2.4 BLEU points. |